7-A. Text Summarization with Sentiment Topic Modeling: BERT (Bidirectional Encoder Representations from Transformers)¶
In [1]:
# pip install bertopic
# pip install sentence-transformers[cpu]
# pip install matplotlib plotly
# pip install gcsfs
# pip install ipywidgets
In [2]:
import os
import time
import math
import re
import sys
import requests
import multiprocessing
from pandarallel import pandarallel
from google.cloud import storage
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from bertopic import BERTopic
from wordcloud import WordCloud
import nltk as nltk
import ast
import os
os.environ["CUDA_VISIBLE_DEVICES"] = ""
os.environ["TOKENIZERS_PARALLELISM"] = "false"
import warnings
# Suppress warnings if necessary
warnings.simplefilter('once')
warnings.simplefilter('ignore')
warnings.filterwarnings("ignore", category=FutureWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)
warnings.filterwarnings(action='ignore', category=UserWarning, module='gensim')
2023-12-06 06:33:13.505400: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2023-12-06 06:33:13.505473: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2023-12-06 06:33:13.509776: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2023-12-06 06:33:13.530378: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
In [3]:
pd.set_option('display.max_rows', 100)
pd.set_option('display.max_columns', None)
pd.set_option('display.max_colwidth', 500)
In [4]:
num_processors = multiprocessing.cpu_count()
num_processors
workers = num_processors-1
print(f'Using {workers} workers')
Using 15 workers
In [5]:
pandarallel.initialize(nb_workers=workers, use_memory_fs=False, progress_bar=True)
INFO: Pandarallel will run on 15 workers. INFO: Pandarallel will use standard multiprocessing data transfer (pipe) to transfer data between the main process and workers.
1. Import Data¶
In [6]:
%%time
file_path = 'news_vader_sent.parquet'
news = pd.read_parquet(file_path)
CPU times: user 24 s, sys: 52.3 s, total: 1min 16s Wall time: 46.7 s
In [7]:
news.shape # (198064, 16)
Out[7]:
(198064, 18)
In [8]:
news.columns
Out[8]:
Index(['url', 'date', 'language', 'title', 'text', 'year', 'month', 'day',
'text_ner', 'text_cleaned', 'text_lemm', 'title_ner', 'title_cleaned',
'title_lemm', 'title_word_count', 'text_word_count', 'vader_sent',
'vader_comp'],
dtype='object')
In [9]:
news.sample(1, random_state = 42)[['text_ner', 'text_cleaned', 'text_lemm', 'title_ner', 'title_cleaned', 'title_lemm']]
Out[9]:
| text_ner | text_cleaned | text_lemm | title_ner | title_cleaned | title_lemm | |
|---|---|---|---|---|---|---|
| 196666 | Prosecutors in all states urge Congress to strengthen tools to fight AI child sexual abuse images Skip to contentCommunity Coverage TourHome ProMedically SpeakingBest of the WestChampions in AgBack to Our AppsCOVID 19Food for NewsTexasNew to a TipLatest CamsClosings and DelaysSend Us Your Weather PhotosTxDOT Highway ConditionsDownload the Weather AppWeather ResourcesKCBD InvestigatesSubmit a TipChad Read ShootingReagor Dykes CoverageSex Trafficking on the South PlainsLubbock County Medical E... | prosecutors states urge congress strengthen tools fight ai child sexual abuse images skip contentcommunity coverage tourhome promedically speakingbest westchampions agback appscovid newstexasnew tiplatest camsclosings delayssend us weather photostxdot highway conditionsdownload weather appweather resourceskcbd investigatessubmit tipchad read shootingreagor dykes coveragesex trafficking south plainslubbock county medical examiner school beat petestats predictionshow watchcommunitytell somethi... | prosecutor state urge congress strengthen tool fight ai child sexual abuse image skip contentcommunity coverage tourhome promedically speakingbest westchampions agback appscovid newstexasnew tiplatest camsclosings delayssend u weather photostxdot highway conditionsdownload weather appweather resourceskcbd investigatessubmit tipchad read shootingreagor dyke coveragesex traffic south plainslubbock county medical examiner school beat petestats predictionshow watchcommunitytell something goodnot... | Prosecutors in all states urge Congress to strengthen tools to fight AI child sexual abuse images | prosecutors states urge congress strengthen tools fight ai child sexual abuse images | prosecutor state urge congress strengthen tool fight ai child sexual abuse image |
2. Sentiment Topic Modeling: BERT¶
Topic modeling (i.e. LDA using gensim or ktrain) or using BERTopic
BERTopic¶
- Nature: BERTopic leverages transformer-based models, like BERT, for generating document embeddings, which capture the contextual relationships between words in a text.
- Methodology: It uses dimensionality reduction (usually UMAP) and clustering algorithms (like HDBSCAN) on top of the embeddings to find topics.
- Advantages: BERTopic excels in capturing the semantic meaning of texts, offering more nuanced and contextually relevant topics.
- Use Cases: It is well-suited for advanced topic modeling tasks where deep contextual understanding is crucial.
- Computational Requirements: Similar to BERT, BERTopic is computationally intensive and generally requires more resources.
LDA in Gensim¶
- Nature: This is a traditional topic modeling approach that assumes each document is a mixture of topics and each topic is a mixture of words.
- Methodology: It uses statistical methods to infer the latent topics in a corpus.
- Advantages: LDA in Gensim is well-established, easy to implement, and less resource-intensive compared to neural network approaches.
- Use Cases: Suitable for basic topic modeling needs where the primary goal is to identify broad topics within a large volume of text.
- Computational Requirements: Can be run efficiently on standard CPU setups.
LDA in ktrain¶
- Nature: ktrain, a wrapper for TensorFlow Keras, simplifies machine learning workflows. Its LDA implementation is similar to Gensim's but integrated within the ktrain ecosystem.
- Methodology: Utilizes statistical methods for topic modeling, akin to Gensim's LDA.
- Advantages: It provides a more user-friendly interface and integrates well with other ktrain functionalities for end-to-end machine learning tasks.
- Use Cases: Ideal for users who prefer a streamlined process for topic modeling along with other machine learning tasks, especially in a Keras/TensorFlow environment.
- Computational Requirements: Comparable to Gensim's LDA in terms of resource needs.
Summary¶
- BERTopic: Best for deep contextual understanding and advanced topic modeling, but resource-intensive.
- LDA in Gensim: A standard, widely-used method for topic modeling, balancing performance and computational efficiency.
- LDA in ktrain: Offers a more accessible and integrated approach within the ktrain framework, suitable for those working within a Keras/TensorFlow environment.
In [10]:
%%time
news['text_tokens'] = news['text_lemm'].parallel_apply(nltk.word_tokenize)
VBox(children=(HBox(children=(IntProgress(value=0, description='0.00%', max=13205), Label(value='0 / 13205')))…
CPU times: user 29.4 s, sys: 25.2 s, total: 54.6 s Wall time: 2min 3s
2.2. BERTopic on Negative Topics¶
In [11]:
news_ne = news[news['vader_sent'] == 'negative']
In [12]:
news_ne.info()
<class 'pandas.core.frame.DataFrame'> Index: 9947 entries, 40 to 198047 Data columns (total 19 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 url 9947 non-null object 1 date 9947 non-null datetime64[ns] 2 language 9947 non-null object 3 title 9947 non-null object 4 text 9947 non-null object 5 year 9947 non-null int32 6 month 9947 non-null int32 7 day 9947 non-null int32 8 text_ner 9947 non-null object 9 text_cleaned 9947 non-null object 10 text_lemm 9947 non-null object 11 title_ner 9947 non-null object 12 title_cleaned 9947 non-null object 13 title_lemm 9947 non-null object 14 title_word_count 9947 non-null int64 15 text_word_count 9947 non-null int64 16 vader_sent 9947 non-null object 17 vader_comp 9947 non-null float64 18 text_tokens 9947 non-null object dtypes: datetime64[ns](1), float64(1), int32(3), int64(2), object(12) memory usage: 1.4+ MB
In [52]:
%%time
mod_BERT_neg = BERTopic(calculate_probabilities=True, verbose=True)
topics_neg, probabilities_neg = mod_BERT_neg.fit_transform(news_ne['text_ner'].tolist())
2023-12-06 07:19:53,736 - BERTopic - Embedding - Transforming documents to embeddings.
Batches: 0%| | 0/311 [00:00<?, ?it/s]
2023-12-06 07:24:57,946 - BERTopic - Embedding - Completed ✓ 2023-12-06 07:24:57,947 - BERTopic - Dimensionality - Fitting the dimensionality reduction algorithm 2023-12-06 07:25:11,530 - BERTopic - Dimensionality - Completed ✓ 2023-12-06 07:25:11,532 - BERTopic - Cluster - Start clustering the reduced embeddings 2023-12-06 07:25:43,314 - BERTopic - Cluster - Completed ✓ 2023-12-06 07:25:43,321 - BERTopic - Representation - Extracting topics from clusters using representation models. 2023-12-06 07:26:02,665 - BERTopic - Representation - Completed ✓
CPU times: user 44min 33s, sys: 8min 3s, total: 52min 37s Wall time: 6min 19s
In [53]:
mod_BERT_neg.get_topic_info().head(20)
Out[53]:
| Topic | Count | Name | Representation | Representative_Docs | |
|---|---|---|---|---|---|
| 0 | -1 | 2234 | -1_the_and_to_of | [the, and, to, of, in, ai, is, that, for, on] | [EXPLAINER What is ChatGPT and why are schools blocking it KWKT FOX Skip to content KWKT FOX Waco Sign Up Waco Sponsored By Toggle Menu Open Navigation Close Navigation Search Please enter a search term. Primary Menu Now Streaming NewsNation Now Live Weather Camera Views by SkyTracker News Local News State News Texas Governor s Debate National and World News Political News Politics from The Hill Washington DC Business News Crime Press Releases Weird News Entertainment News Health News Corona... |
| 1 | 0 | 175 | 0_jones_florida_rebekah_she | [jones, florida, rebekah, she, warrant, state, her, covid, fired, scientist] | [Ex Florida data scientist Rebekah Jones plans to surrender Skip to Local BusinessLive StreamingLivestream CamsWeather BlogGood Morning Black Hills60 Second CalendarContact UsMeet the TeamStation JobsPhotosSubmit a StoryProgramming ScheduleCOVID Local BusinessesFull Court Press with Greta Van SusterenCircle Country Music LifestyleGray DC BureauInvestigate TVPowerNationLatest Florida data scientist Rebekah Jones plans to surrenderThe Florida Department of Law Enforcement said Rebekah Jones ha... |
| 2 | 1 | 142 | 1_students_chatgpt_exam_cheating | [students, chatgpt, exam, cheating, university, student, school, schools, it, education] | [EXPLAINER What is ChatGPT and why are schools blocking it Skip to main content Beaumont Enterprise Homepage Currently Reading EXPLAINER What is ChatGPT and why are schools blocking it Subscribe Subscribe e Edition Sign In Servicese DirectoryAbout SafetyHurricane Content SchoolLamarHouston AstrosHouston RocketsHouston DirectorySponsored Content DécorSETX Real EstateOutdoors Recommended Entergy helps fight hunger in Southeast Texas TX Gulf Coast fishing, possible state regulation changes in B... |
| 3 | 2 | 114 | 2_italy_openai_data_chatgpt | [italy, openai, data, chatgpt, italian, protection, privacy, users, watchdog, ban] | [Italy temporarily blocks ChatGPT over privacy concerns Today In BC SearchHome Newsletters Subscribe Subscribe Login Logout Support Centre Puzzles Contests News COVID B.C Politics National Politics World News Sports Cannabis Travel Podcasts Video Opinion Classifieds Jobs Business Entertainment Life Weather Obituaries Contact Us Contact Us Black Press FAQ Privacy Policy Terms of use News B.C Politics National Cannabis Travel Obituaries Classifieds Contact Us Subscribe Login Puzzles Contests P... |
| 4 | 3 | 110 | 3_humanity_ai_extinction_could | [humanity, ai, extinction, could, risks, humans, risk, nuclear, intelligence, be] | [Artificial Intelligence Raises Risk of Extinction, Experts Say in New Warning NBC10 Philadelphia Skip to content Main Navigation Search Search for Weather Local Sports Entertainment Investigators Videos Newsletters Live TV Share Close Trending Watch NBC10 on Streaming Platforms Wawa Welcome America Philly Mayoral Race Phillies Baseball Expand artificial intelligence Artificial Intelligence Raises Risk of Extinction, Experts Say in New Warning Worries about artificial intelligence systems ou... |
| 5 | 4 | 100 | 4_bard_google_being_chatbot | [bard, google, being, chatbot, or, chatgpt, may, stereotyped, people, less] | [Google has long dominated search, and it s about to massively change it to meet the rising threat of ChatGPT. New AI features could intensify battles with content creators. NewsBreakSign ArtTV SeriesBooks DanceBehind Viral VideosPerforming ArtsTV MusicHip. HealthHealth ServicesMental HealthDiseases s HealthCancerFood SportsPremier DrinksPetsBeauty SafetyPublic SafetyAccidentsLaw EnforcementTraffic AdviceFamily RentLabor IssuesTrouble ScienceEarth NationsMiddle locations, channels, topics, p... |
| 6 | 5 | 95 | 5_fraud_financial_complyadvantage_resistant | [fraud, financial, complyadvantage, resistant, risk, banking, transaction, transactions, banks, detection] | [Resistant AI and ComplyAdvantage Launch AI Transaction Monitoring Solution To Combat Fraud and Money Laundering Skip to contentFirst Alert Photos PoliticsFirst Alert CamerasRadarRiver LevelsRoad Conditions7 Day ForecastWeather MapsSportsHigh SchoolCMUAthlete Of The WeekJUCO World SeriesElection ResultsNational Results MapContact UsMeet the TeamFind Our ChannelJUCO LiveContestsMovies Under the Up For E NewsLIVECW13Job BoardStation JobsCOVID MapSubmit Photos VideoProgramming ScheduleGo to the... |
| 7 | 6 | 92 | 6_india_delhi_opindia_modi | [india, delhi, opindia, modi, indian, vs, february, viral, in, check] | [Nvidia CEO Jensen Huang meets PM Modi, highlights growing partnership in AI and India s young talent pool Politics Opinions Fact Check Media Fact Check Political Fact Check Social Media Fact Check Media Variety Books Culture and History Satire WTF News Specials Interviews OpIndia Explains OpIndia Scoops More Social Media Entertainment Political History of India Government and Policy Economy and Finance Sports World Crime Law Support Us ह न द म ગ જર ત મ Search OpIndiabringing the right side ... |
| 8 | 7 | 90 | 7_fox_county_ai_news | [fox, county, ai, news, public, atlanta, that, of, and, filefcc] | [Misinformation machines AI chatbots can spew falsehoods, even accuse people of crimes they never committed LiveNewsWeatherGood DayTrafficContests More Watch Live Expand Collapse search Search site News Local NewsNational FinanceSportsFOX News SundayFOX Live InstaPollWeather ForecastFOX DelaysFOX Weather AppFOX WeatherGood Day LION Lunch HourCooking With ComoLights Camera McCarthy DMV DestinationsPay It ForwardZip TripPodcasts Shows LION Lunch HourDMV ZoneLike It Or Not The Final 5On The Hil... |
| 9 | 8 | 90 | 8_breast_cancer_radiologists_screening | [breast, cancer, radiologists, screening, mammograms, women, doctors, risk, mammogram, cancers] | [Artificial Intelligence Is Helping Improve Breast Cancer Detection For Women With Dense Breasts Tech Times Tech Science Business Health Culture Features Buzz Home Health Health Public Living Wellness Artificial Intelligence Is Helping Improve Breast Cancer Detection For Women With Dense Breasts Facebook facebook Twitter twitter Reddit reddit Comment LinkedIn linkedin Email Email WhatsApp What s app RJ Pierce, Tech Times October, am An artificial intelligence algorithm has been helping impro... |
| 10 | 9 | 90 | 9_images_trump_higgins_midjourney | [images, trump, higgins, midjourney, image, putin, fake, context, were, donald] | [Trump arrested Putin jailed Fake AI images spread online Skip to NebraskaContact UsWatch VideoWatch RadarWatch WeatherWeather Shield Request FormWeather CamerasDownload the Weather SchoolSports ResultsNational Politics1011 CaresPure NebraskaPure Nebraska VideoContestsCan Care A VanContact UsMeet the TeamSubmit a News TipCareersAdvertise With Us10 WeatherLocal EverydayHealthy EverydaySeniors EverydayProgramming ScheduleSubmit Photos and VideosCircle Country Music LifestyleGray DC BureauInves... |
| 11 | 10 | 87 | 10_gebru_google_mitchell_timnit | [gebru, google, mitchell, timnit, researcher, company, fired, ethics, her, firing] | [Google fires AI researcher, one of few Black woman in field Stuff.co.nz in TiakiSpotlightStuff NationCartoonsKEA Kids of PlentyTaranakihawke s familyPlay eventsstuff fires AI researcher, one of few Black woman in fieldMatt O Brien14, Dec White Getty ImagesProminent artificial intelligence scholar Timnit Gebru helped improve Google s public image as a company that elevates Black computer scientists and questions harmful uses of AI artificial intelligence scholar Timnit Gebru helped improve G... |
| 12 | 11 | 81 | 11_suicide_safetylit_suicidal_aa | [suicide, safetylit, suicidal, aa, machine, learning, update, doi, bulletin, pdf] | [SafetyLit A comparative study of machine learning techniques for suicide attempts predictive model HOME SEARCH BOOLEAN SEARCH THESAURUS SOURCES AUTHORS WEEKLY UPDATES UPDATE BULLETIN PDF UPDATE BULLETIN WEB ABOUT US MY SAFETYLIT Aa Aa Aa SAFETYLIT WEEKLY UPDATE We compile citations and summaries of about new articles every week. Email Signup RSS Feed HELP Tutorials FAQ CONTACT US Contact info SafetyLit is a service of Search Results Journal Article A comparative study of machine learning te... |
| 13 | 12 | 80 | 12_clearview_facial_recognition_privacy | [clearview, facial, recognition, privacy, enforcement, database, company, law, biometric, shelagh] | [If you re worried about the end of privacy, don t waste your outrage on Clearview AI VentureBeat VentureBeat Homepage Channels GamesBeat AI AR VR Big Data Business Cloud Commerce Dev Enterprise Entrepreneur Esports Marketing Media Mobile PC Gaming Security Social Transportation Got a news tip Press Releases Webinars Advertise VB Lab Guest Posts Events Upcoming Transform GB Summit Sponsor Media Partner Got a news tip Press Releases Webinars Advertise VB Lab Guest Posts Newsletters Special Is... |
| 14 | 13 | 78 | 13_india_flight_air_express | [india, flight, air, express, kozhikode, passengers, plane, airport, delhi, runway] | [Kozhikode plane crash injured passengers discharged from hospitals, says AI Express Select EditionPacificSouth AsiaEast and South East AsiaEurope and Central AsiaCentral AfricaEast AfricaSouthern Africa West AfricaMiddle East and North AfricaNorth America Latin America and Caribbean HOME NEWS RESEARCH LIVE DISCOURSE BLOG OPINION SUBMIT PRESS RELEASE About Career Advertisement Team Partnership Knowledge Partnership Media Partnership Contact Us NEWS RESEARCH LIVE DISCOURSE BLOG OPINION INTERV... |
| 15 | 14 | 78 | 14_number_markets_datanetchng_dataprice | [number, markets, datanetchng, dataprice, canada, market, fed, bank, bloomberg, inflation] | [Nvidia s Surge Sharpens Focus on Hunt for AI Losers BNN Bloomberg Markets Indices Currencies Energy Metals data.symbol group.RICS data.netChng number data.netChng number data data.price number data.price number data.symbol group.RICS data.netChng number data.netChng number data data.price number data.price number Markets As of timeStamp.date timeStamp.time Markets Markets Indices Currencies Energy Metals data.symbol group.RICS data.netChng number data.netChng number data data.price number d... |
| 16 | 15 | 74 | 15_safetylit_aa_update_severity | [safetylit, aa, update, severity, doi, pdf, machine, learning, bulletin, crashes] | [SafetyLit The chance of a traffic collision is predicted using machine learning HOME SEARCH BOOLEAN SEARCH THESAURUS SOURCES AUTHORS WEEKLY UPDATES UPDATE BULLETIN PDF UPDATE BULLETIN WEB ABOUT US MY SAFETYLIT Aa Aa Aa SAFETYLIT WEEKLY UPDATE We compile citations and summaries of about new articles every week. RSS Feed HELP Tutorials FAQ CONTACT US Contact info SafetyLit is a service of Search Results Journal Article The chance of a traffic collision is predicted using machine learning Cita... |
| 17 | 16 | 71 | 16_images_child_abuse_sexual | [images, child, abuse, sexual, porn, deepfake, children, csam, nude, girls] | [Deepfake porn could be a growing problem amid AI race WTTV CBS4Indy Skip to content WTTV CBS4Indy Indianapolis WATCH NOW CBS4 News Sign Up Indianapolis WATCH NOW Sponsored By Toggle Menu Open Navigation Close Navigation Search Please enter a search term. Primary Menu News Indiana News Video Indianapolis Area Crime Consumer News Consumer Alerts Black History Month Your Local Election Headquarters Politics from The Hill NewsNation Now National and World Viral Your Health BestReviews BestRevie... |
| 18 | 17 | 70 | 17_chevron_cnn_oregon_idaho | [chevron, cnn, oregon, idaho, ktvz, bend, local, button, community, weather] | [Chinese police detain man for allegedly using ChatGPT to spread rumors online KTVZ circle arrow Play Button Stop Button chevron right chevron left chevron up search warning chevron left skinny chevron right skinny x clock calendar play button cancel circle user twitter facebook youtube instagram email linkedin Home News Central Oregon Oregon Northwest US World COVID Fire Alert Crime Stoppers KTVZ.COM Polls Special Reports Weather Alerts Interactive Radar Local Forecast Snow Report Road Cond... |
| 19 | 18 | 68 | 18_china_chinese_chips_chip | [china, chinese, chips, chip, beijing, taiwan, us, its, nvidia, alibaba] | [Pull US AI Research Out of China NewsBreakSearch locations, channels, topics, people ... inSign CHANNELSAdd of UsePrivacy PolicyDo Not Sell My InfoHelp CenterAbout Particle Media.RELATED PEOPLEPersonXi JinpingPersonBen SasseIN THIS ARTICLE Google Ai Censorship In China Us Ai Microsoft Exchange Chinese Ccp Georgetown University Facebook Google Ibm Research Asia Lab Party The Joint Chiefs Of Staff Seagate Hikvision American AiYOU MAY ALSO, CNNews of UsePrivacy PolicyDo Not Sell My InfoHelp Ce... |
In [54]:
negative_topic_df = pd.DataFrame(mod_BERT_neg.get_topic_info())
In [55]:
print(negative_topic_df.shape)
(253, 5)
In [56]:
negative_topic_df.head()
Out[56]:
| Topic | Count | Name | Representation | Representative_Docs | |
|---|---|---|---|---|---|
| 0 | -1 | 2234 | -1_the_and_to_of | [the, and, to, of, in, ai, is, that, for, on] | [EXPLAINER What is ChatGPT and why are schools blocking it KWKT FOX Skip to content KWKT FOX Waco Sign Up Waco Sponsored By Toggle Menu Open Navigation Close Navigation Search Please enter a search term. Primary Menu Now Streaming NewsNation Now Live Weather Camera Views by SkyTracker News Local News State News Texas Governor s Debate National and World News Political News Politics from The Hill Washington DC Business News Crime Press Releases Weird News Entertainment News Health News Corona... |
| 1 | 0 | 175 | 0_jones_florida_rebekah_she | [jones, florida, rebekah, she, warrant, state, her, covid, fired, scientist] | [Ex Florida data scientist Rebekah Jones plans to surrender Skip to Local BusinessLive StreamingLivestream CamsWeather BlogGood Morning Black Hills60 Second CalendarContact UsMeet the TeamStation JobsPhotosSubmit a StoryProgramming ScheduleCOVID Local BusinessesFull Court Press with Greta Van SusterenCircle Country Music LifestyleGray DC BureauInvestigate TVPowerNationLatest Florida data scientist Rebekah Jones plans to surrenderThe Florida Department of Law Enforcement said Rebekah Jones ha... |
| 2 | 1 | 142 | 1_students_chatgpt_exam_cheating | [students, chatgpt, exam, cheating, university, student, school, schools, it, education] | [EXPLAINER What is ChatGPT and why are schools blocking it Skip to main content Beaumont Enterprise Homepage Currently Reading EXPLAINER What is ChatGPT and why are schools blocking it Subscribe Subscribe e Edition Sign In Servicese DirectoryAbout SafetyHurricane Content SchoolLamarHouston AstrosHouston RocketsHouston DirectorySponsored Content DécorSETX Real EstateOutdoors Recommended Entergy helps fight hunger in Southeast Texas TX Gulf Coast fishing, possible state regulation changes in B... |
| 3 | 2 | 114 | 2_italy_openai_data_chatgpt | [italy, openai, data, chatgpt, italian, protection, privacy, users, watchdog, ban] | [Italy temporarily blocks ChatGPT over privacy concerns Today In BC SearchHome Newsletters Subscribe Subscribe Login Logout Support Centre Puzzles Contests News COVID B.C Politics National Politics World News Sports Cannabis Travel Podcasts Video Opinion Classifieds Jobs Business Entertainment Life Weather Obituaries Contact Us Contact Us Black Press FAQ Privacy Policy Terms of use News B.C Politics National Cannabis Travel Obituaries Classifieds Contact Us Subscribe Login Puzzles Contests P... |
| 4 | 3 | 110 | 3_humanity_ai_extinction_could | [humanity, ai, extinction, could, risks, humans, risk, nuclear, intelligence, be] | [Artificial Intelligence Raises Risk of Extinction, Experts Say in New Warning NBC10 Philadelphia Skip to content Main Navigation Search Search for Weather Local Sports Entertainment Investigators Videos Newsletters Live TV Share Close Trending Watch NBC10 on Streaming Platforms Wawa Welcome America Philly Mayoral Race Phillies Baseball Expand artificial intelligence Artificial Intelligence Raises Risk of Extinction, Experts Say in New Warning Worries about artificial intelligence systems ou... |
In [57]:
negative_topic_df.to_parquet('bert_ne_topic_info.parquet')
In [58]:
# Google Cloud Storage details
bucket_name = 'nlp-final'
file_path = 'bert_ne_topic_info.parquet' # This is the name the file will have in GCS
local_file_path = 'bert_ne_topic_info.parquet' # Path to the local file you just saved
# Create a GCS Client
storage_client = storage.Client()
# Get the bucket
bucket = storage_client.get_bucket(bucket_name)
# Create a blob object from the filepath
blob = bucket.blob(file_path)
# Upload the file
blob.upload_from_filename(local_file_path)
In [59]:
news_ne['bert_topics'] = mod_BERT_neg.topics_
# news_ne['bert_topics_words'] = news_ne['bert_topics'].apply(lambda x: mod_BERT_neg.get_topic(x))
In [60]:
news_ne.sample(3, random_state = 42)
Out[60]:
| url | date | language | title | text | year | month | day | text_ner | text_cleaned | text_lemm | title_ner | title_cleaned | title_lemm | title_word_count | text_word_count | vader_sent | vader_comp | text_tokens | bert_topics | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 72547 | https://www.newschannel10.com/2021/01/18/ex-florida-data-scientist-jail-after-arrest-warrant-issued/ | 2021-01-19 | en | Ex-Florida data scientist turns herself in after arrest warrant issued | Ex-Florida data scientist turns herself in after arrest warrant issued \n\n \n\n Skip to content Go Local Grow with Us Expert Connections Health Connections Contests Moms Talk Baby Boomers Talk Panhandle Deals Viewers Choice Awards Home News WATCH LIVE Weather Closings Coronavirus Vaccine Watch Community Sports About Us Home Election Res... | 2021 | 1 | 19 | Ex Florida data scientist turns herself in after arrest warrant issued Skip to content Go Local Grow with Us Expert Connections Health Connections Contests Moms Talk Baby Boomers Talk Panhandle Deals Viewers Choice Awards Home News WATCH LIVE Weather Closings Coronavirus Vaccine Watch Community Sports About Us Home Election Results Download our Apps WATCH LIVE Go Local News National Crime Education Perspective with Brent McClure Good News With Doppler Dave Coronavirus Vaccine Watch Panhandle... | ex florida data scientist turns arrest warrant issued skip content go local grow us expert connections health connections contests moms talk baby boomers talk panhandle deals viewers choice awards home news watch live weather closings coronavirus vaccine watch community sports us home election results download apps watch live go local news national crime education perspective brent mcclure good news doppler dave coronavirus vaccine watch panhandle magazine winter summer spring winter fall us... | ex florida data scientist turn arrest warrant issue skip content go local grow u expert connection health connection contest mom talk baby boomer talk panhandle deal viewer choice award home news watch live weather closing coronavirus vaccine watch community sport u home election result download apps watch live go local news national crime education perspective brent mcclure good news doppler dave coronavirus vaccine watch panhandle magazine winter summer spring winter fall u advertise newsc... | Ex Florida data scientist turns herself in after arrest warrant issued | ex florida data scientist turns arrest warrant issued | ex florida data scientist turn arrest warrant issue | 8 | 613 | negative | -0.9840 | [ex, florida, data, scientist, turn, arrest, warrant, issue, skip, content, go, local, grow, u, expert, connection, health, connection, contest, mom, talk, baby, boomer, talk, panhandle, deal, viewer, choice, award, home, news, watch, live, weather, closing, coronavirus, vaccine, watch, community, sport, u, home, election, result, download, apps, watch, live, go, local, news, national, crime, education, perspective, brent, mcclure, good, news, doppler, dave, coronavirus, vaccine, watch, panh... | 0 |
| 105006 | https://www.mysanantonio.com/news/article/Flood-forecasts-in-real-time-with-block-by-block-17725033.php | 2023-01-18 | en | Flood forecasts in real-time with block-by-block data could save lives – a new machine learning method makes it possible | \nFlood forecasts in real-time with block-by-block data could save lives – a new machine learning method makes it possible\n\n \n\n \n \n\n \n\n \n\n \n \n\n \nSkip to main content\n\n\n MySA Homepage\n\nCurrently Reading\nFlood forecasts in real-time with block-by-block data could save lives – a new machine learning method makes it possible\n\nNewsletters\n\nSign In\n\n \nHomeSubscribeBuy E-N MerchandiseContact UsAbout UsAdvertise With UsPlace a Classified AdPrivacy NoticeNewsletters & Tex... | 2023 | 1 | 18 | Flood forecasts in real time with block by block data could save lives a new machine learning method makes it possible Skip to main content MySA Homepage Currently Reading Flood forecasts in real time with block by block data could save lives a new machine learning method makes it possible Newsletters Sign In HomeSubscribeBuy E N MerchandiseContact UsAbout UsAdvertise With UsPlace a Classified AdPrivacy NoticeNewsletters Text AlertsFind a Business in S.A.Manage by to San AntonioClassified Ma... | flood forecasts real time block block data could save lives new machine learning method makes possible skip main content mysa homepage currently reading flood forecasts real time block block data could save lives new machine learning method makes possible newsletters sign homesubscribebuy merchandisecontact usabout usadvertise usplace classified adprivacy noticenewsletters text alertsfind business san antonioclassified marketplacetop lawyersnationborder newsreal estatehome searchland salesre... | flood forecast real time block block data could save life new machine learn method make possible skip main content mysa homepage currently reading flood forecast real time block block data could save life new machine learn method make possible newsletter sign homesubscribebuy merchandisecontact usabout usadvertise usplace classify adprivacy noticenewsletters text alertsfind business san antonioclassified marketplacetop lawyersnationborder newsreal estatehome searchland salesrentalshomes guid... | Flood forecasts in real time with block by block data could save lives | flood forecasts real time block block data could save lives | flood forecast real time block block data could save life | 10 | 964 | negative | -0.1725 | [flood, forecast, real, time, block, block, data, could, save, life, new, machine, learn, method, make, possible, skip, main, content, mysa, homepage, currently, reading, flood, forecast, real, time, block, block, data, could, save, life, new, machine, learn, method, make, possible, newsletter, sign, homesubscribebuy, merchandisecontact, usabout, usadvertise, usplace, classify, adprivacy, noticenewsletters, text, alertsfind, business, san, antonioclassified, marketplacetop, lawyersnationbord... | -1 |
| 107850 | https://www.knoe.com/2023/03/29/musk-scientists-call-halt-ai-race-sparked-by-chatgpt/ | 2023-03-29 | en | Musk, scientists call for halt to AI race sparked by ChatGPT | Musk, scientists call for halt to AI race sparked by ChatGPT\n\nSkip to contentTornado Disaster ReliefNewsWeatherSportsOur TownLivestreamContestsNewsArkansasCOVID-19 InfoWhat's Your StoryNationalRegionalStateNELA Home ShowWeatherWeather MapsRadarWeather BlogWeather AcademyWeather RadioSevere Weather ResourcesClosingsLivestreamSportsLocal ScoresBeat the AceTeam of the WeekAaron's AcesCheerleader ChallengeCommunity CalendarContestsCOVID-19 MapGood Morning ArkLaMissGuest RecipesGuest Interview ... | 2023 | 3 | 29 | Musk, scientists call for halt to AI race sparked by ChatGPT Skip to contentTornado Disaster InfoWhat s Your Home ShowWeatherWeather MapsRadarWeather BlogWeather AcademyWeather RadioSevere Weather ScoresBeat the AceTeam of the WeekAaron s AcesCheerleader ChallengeCommunity MapGood Morning ArkLaMissGuest RecipesGuest Interview Request FormHealth ConnectionsPerfect HomeOur TownService SaluteSubmit Photos and VideosFeed Your SoulRecommend Your Favorite RestaurantMr. FoodTalking FoodTV ListingsS... | musk scientists call halt ai race sparked chatgpt skip contenttornado disaster infowhat home showweatherweather mapsradarweather blogweather academyweather radiosevere weather scoresbeat aceteam weekaaron acescheerleader challengecommunity mapgood morning arklamissguest recipesguest interview request formhealth connectionsperfect homeour townservice salutesubmit photos videosfeed soulrecommend favorite restaurantmr foodtalking foodtv listingsstation jobscontact usmeet teamadvertise usjobsclo... | musk scientist call halt ai race spark chatgpt skip contenttornado disaster infowhat home showweatherweather mapsradarweather blogweather academyweather radiosevere weather scoresbeat aceteam weekaaron acescheerleader challengecommunity mapgood morning arklamissguest recipesguest interview request formhealth connectionsperfect homeour townservice salutesubmit photo videosfeed soulrecommend favorite restaurantmr foodtalking foodtv listingsstation jobscontact usmeet teamadvertise usjobsclosed ... | Musk, scientists call for halt to AI race sparked by ChatGPT | musk scientists call halt ai race sparked chatgpt | musk scientist call halt ai race spark chatgpt | 8 | 533 | negative | -0.2247 | [musk, scientist, call, halt, ai, race, spark, chatgpt, skip, contenttornado, disaster, infowhat, home, showweatherweather, mapsradarweather, blogweather, academyweather, radiosevere, weather, scoresbeat, aceteam, weekaaron, acescheerleader, challengecommunity, mapgood, morning, arklamissguest, recipesguest, interview, request, formhealth, connectionsperfect, homeour, townservice, salutesubmit, photo, videosfeed, soulrecommend, favorite, restaurantmr, foodtalking, foodtv, listingsstation, jo... | -1 |
Topic Visualization¶
In [61]:
fig = mod_BERT_neg.visualize_topics()
fig.write_html("bertopic_visualization.html") # For saving as interactive HTML
fig.show()
Topic Frequency¶
In [62]:
fig = mod_BERT_neg.visualize_barchart()
fig.write_html("topic_frequency.html")
Topic Hierarchy¶
In [63]:
fig = mod_BERT_neg.visualize_hierarchy()
fig.write_html("topic_hierarchy.html")
Topic Similarity¶
In [64]:
fig = mod_BERT_neg.visualize_heatmap()
fig.write_html("topic_similarity.html")
Intertopic Distance Map¶
In [65]:
fig = mod_BERT_neg.visualize_topics()
fig.write_html("intertopic_distance_map.html")
In [66]:
print("Number of topics:", mod_BERT_neg.get_topic_freq().shape[0])
Number of topics: 253
In [67]:
news_ne.to_parquet('news_bert_ne.parquet')
In [68]:
# Google Cloud Storage details
bucket_name = 'nlp-final'
file_path = 'news_bert_ne.parquet' # This is the name the file will have in GCS
local_file_path = 'news_bert_ne.parquet' # Path to the local file you just saved
# Create a GCS Client
storage_client = storage.Client()
# Get the bucket
bucket = storage_client.get_bucket(bucket_name)
# Create a blob object from the filepath
blob = bucket.blob(file_path)
# Upload the file
blob.upload_from_filename(local_file_path)
In [69]:
%%time
file_path = 'news_bert_ne.parquet'
news_ne = pd.read_parquet(file_path)
CPU times: user 1.85 s, sys: 676 ms, total: 2.52 s Wall time: 1.46 s
In [70]:
%%time
file_path = 'bert_ne_topic_info.parquet'
negative_topic_df = pd.read_parquet(file_path)
CPU times: user 30.5 ms, sys: 12.4 ms, total: 42.8 ms Wall time: 36.1 ms
3. Negative Sentiment Analysis Overtime¶
3.1. Understanding the Main Topics¶
1. Topic Distribution¶
In [71]:
%%time
file_path = 'news_bert_ne.parquet'
news_ne = pd.read_parquet(file_path)
CPU times: user 1.89 s, sys: 563 ms, total: 2.45 s Wall time: 1.48 s
In [72]:
%%time
file_path = 'bert_ne_topic_info.parquet'
negative_topic_df = pd.read_parquet(file_path)
CPU times: user 46.2 ms, sys: 19.8 ms, total: 66 ms Wall time: 53.2 ms
In [73]:
news_ne[['text_ner', 'bert_topics']].sample(3, random_state = 42)
Out[73]:
| text_ner | bert_topics | |
|---|---|---|
| 72547 | Ex Florida data scientist turns herself in after arrest warrant issued Skip to content Go Local Grow with Us Expert Connections Health Connections Contests Moms Talk Baby Boomers Talk Panhandle Deals Viewers Choice Awards Home News WATCH LIVE Weather Closings Coronavirus Vaccine Watch Community Sports About Us Home Election Results Download our Apps WATCH LIVE Go Local News National Crime Education Perspective with Brent McClure Good News With Doppler Dave Coronavirus Vaccine Watch Panhandle... | 0 |
| 105006 | Flood forecasts in real time with block by block data could save lives a new machine learning method makes it possible Skip to main content MySA Homepage Currently Reading Flood forecasts in real time with block by block data could save lives a new machine learning method makes it possible Newsletters Sign In HomeSubscribeBuy E N MerchandiseContact UsAbout UsAdvertise With UsPlace a Classified AdPrivacy NoticeNewsletters Text AlertsFind a Business in S.A.Manage by to San AntonioClassified Ma... | -1 |
| 107850 | Musk, scientists call for halt to AI race sparked by ChatGPT Skip to contentTornado Disaster InfoWhat s Your Home ShowWeatherWeather MapsRadarWeather BlogWeather AcademyWeather RadioSevere Weather ScoresBeat the AceTeam of the WeekAaron s AcesCheerleader ChallengeCommunity MapGood Morning ArkLaMissGuest RecipesGuest Interview Request FormHealth ConnectionsPerfect HomeOur TownService SaluteSubmit Photos and VideosFeed Your SoulRecommend Your Favorite RestaurantMr. FoodTalking FoodTV ListingsS... | -1 |
In [74]:
news_ne['bert_topics'].value_counts(ascending = False).reset_index(name = 'count')
Out[74]:
| bert_topics | count | |
|---|---|---|
| 0 | -1 | 2234 |
| 1 | 0 | 175 |
| 2 | 1 | 142 |
| 3 | 2 | 114 |
| 4 | 3 | 110 |
| ... | ... | ... |
| 248 | 247 | 11 |
| 249 | 248 | 10 |
| 250 | 249 | 10 |
| 251 | 250 | 10 |
| 252 | 251 | 10 |
253 rows × 2 columns
In [75]:
news_ne['bert_topics'].value_counts(ascending = False, normalize = True).reset_index(name = 'portion')
Out[75]:
| bert_topics | portion | |
|---|---|---|
| 0 | -1 | 0.224590 |
| 1 | 0 | 0.017593 |
| 2 | 1 | 0.014276 |
| 3 | 2 | 0.011461 |
| 4 | 3 | 0.011059 |
| ... | ... | ... |
| 248 | 247 | 0.001106 |
| 249 | 248 | 0.001005 |
| 250 | 249 | 0.001005 |
| 251 | 250 | 0.001005 |
| 252 | 251 | 0.001005 |
253 rows × 2 columns
2. Topic related information: Interpretation¶
- Topic: Each topic is typically assigned a unique identifier (an integer). Special attention should be paid to topic -1, as it often represents outliers or miscellaneous text.
- Count: This indicates the number of documents associated with each topic. Topics with a high count are more prevalent in your dataset.
- Name: Generated based on the most frequent and representative words of each topic. These names give a quick idea of what the topic is about.
- Representation: Shows key words that are characteristic of the topic.
- Representative_Docs: Provides documents (or parts of them) that are most representative of the topic. These can be used to understand the context in which the topic keywords appear.
In [76]:
negative_topic_df.head(20)
Out[76]:
| Topic | Count | Name | Representation | Representative_Docs | |
|---|---|---|---|---|---|
| 0 | -1 | 2234 | -1_the_and_to_of | [the, and, to, of, in, ai, is, that, for, on] | [EXPLAINER What is ChatGPT and why are schools blocking it KWKT FOX Skip to content KWKT FOX Waco Sign Up Waco Sponsored By Toggle Menu Open Navigation Close Navigation Search Please enter a search term. Primary Menu Now Streaming NewsNation Now Live Weather Camera Views by SkyTracker News Local News State News Texas Governor s Debate National and World News Political News Politics from The Hill Washington DC Business News Crime Press Releases Weird News Entertainment News Health News Corona... |
| 1 | 0 | 175 | 0_jones_florida_rebekah_she | [jones, florida, rebekah, she, warrant, state, her, covid, fired, scientist] | [Ex Florida data scientist Rebekah Jones plans to surrender Skip to Local BusinessLive StreamingLivestream CamsWeather BlogGood Morning Black Hills60 Second CalendarContact UsMeet the TeamStation JobsPhotosSubmit a StoryProgramming ScheduleCOVID Local BusinessesFull Court Press with Greta Van SusterenCircle Country Music LifestyleGray DC BureauInvestigate TVPowerNationLatest Florida data scientist Rebekah Jones plans to surrenderThe Florida Department of Law Enforcement said Rebekah Jones ha... |
| 2 | 1 | 142 | 1_students_chatgpt_exam_cheating | [students, chatgpt, exam, cheating, university, student, school, schools, it, education] | [EXPLAINER What is ChatGPT and why are schools blocking it Skip to main content Beaumont Enterprise Homepage Currently Reading EXPLAINER What is ChatGPT and why are schools blocking it Subscribe Subscribe e Edition Sign In Servicese DirectoryAbout SafetyHurricane Content SchoolLamarHouston AstrosHouston RocketsHouston DirectorySponsored Content DécorSETX Real EstateOutdoors Recommended Entergy helps fight hunger in Southeast Texas TX Gulf Coast fishing, possible state regulation changes in B... |
| 3 | 2 | 114 | 2_italy_openai_data_chatgpt | [italy, openai, data, chatgpt, italian, protection, privacy, users, watchdog, ban] | [Italy temporarily blocks ChatGPT over privacy concerns Today In BC SearchHome Newsletters Subscribe Subscribe Login Logout Support Centre Puzzles Contests News COVID B.C Politics National Politics World News Sports Cannabis Travel Podcasts Video Opinion Classifieds Jobs Business Entertainment Life Weather Obituaries Contact Us Contact Us Black Press FAQ Privacy Policy Terms of use News B.C Politics National Cannabis Travel Obituaries Classifieds Contact Us Subscribe Login Puzzles Contests P... |
| 4 | 3 | 110 | 3_humanity_ai_extinction_could | [humanity, ai, extinction, could, risks, humans, risk, nuclear, intelligence, be] | [Artificial Intelligence Raises Risk of Extinction, Experts Say in New Warning NBC10 Philadelphia Skip to content Main Navigation Search Search for Weather Local Sports Entertainment Investigators Videos Newsletters Live TV Share Close Trending Watch NBC10 on Streaming Platforms Wawa Welcome America Philly Mayoral Race Phillies Baseball Expand artificial intelligence Artificial Intelligence Raises Risk of Extinction, Experts Say in New Warning Worries about artificial intelligence systems ou... |
| 5 | 4 | 100 | 4_bard_google_being_chatbot | [bard, google, being, chatbot, or, chatgpt, may, stereotyped, people, less] | [Google has long dominated search, and it s about to massively change it to meet the rising threat of ChatGPT. New AI features could intensify battles with content creators. NewsBreakSign ArtTV SeriesBooks DanceBehind Viral VideosPerforming ArtsTV MusicHip. HealthHealth ServicesMental HealthDiseases s HealthCancerFood SportsPremier DrinksPetsBeauty SafetyPublic SafetyAccidentsLaw EnforcementTraffic AdviceFamily RentLabor IssuesTrouble ScienceEarth NationsMiddle locations, channels, topics, p... |
| 6 | 5 | 95 | 5_fraud_financial_complyadvantage_resistant | [fraud, financial, complyadvantage, resistant, risk, banking, transaction, transactions, banks, detection] | [Resistant AI and ComplyAdvantage Launch AI Transaction Monitoring Solution To Combat Fraud and Money Laundering Skip to contentFirst Alert Photos PoliticsFirst Alert CamerasRadarRiver LevelsRoad Conditions7 Day ForecastWeather MapsSportsHigh SchoolCMUAthlete Of The WeekJUCO World SeriesElection ResultsNational Results MapContact UsMeet the TeamFind Our ChannelJUCO LiveContestsMovies Under the Up For E NewsLIVECW13Job BoardStation JobsCOVID MapSubmit Photos VideoProgramming ScheduleGo to the... |
| 7 | 6 | 92 | 6_india_delhi_opindia_modi | [india, delhi, opindia, modi, indian, vs, february, viral, in, check] | [Nvidia CEO Jensen Huang meets PM Modi, highlights growing partnership in AI and India s young talent pool Politics Opinions Fact Check Media Fact Check Political Fact Check Social Media Fact Check Media Variety Books Culture and History Satire WTF News Specials Interviews OpIndia Explains OpIndia Scoops More Social Media Entertainment Political History of India Government and Policy Economy and Finance Sports World Crime Law Support Us ह न द म ગ જર ત મ Search OpIndiabringing the right side ... |
| 8 | 7 | 90 | 7_fox_county_ai_news | [fox, county, ai, news, public, atlanta, that, of, and, filefcc] | [Misinformation machines AI chatbots can spew falsehoods, even accuse people of crimes they never committed LiveNewsWeatherGood DayTrafficContests More Watch Live Expand Collapse search Search site News Local NewsNational FinanceSportsFOX News SundayFOX Live InstaPollWeather ForecastFOX DelaysFOX Weather AppFOX WeatherGood Day LION Lunch HourCooking With ComoLights Camera McCarthy DMV DestinationsPay It ForwardZip TripPodcasts Shows LION Lunch HourDMV ZoneLike It Or Not The Final 5On The Hil... |
| 9 | 8 | 90 | 8_breast_cancer_radiologists_screening | [breast, cancer, radiologists, screening, mammograms, women, doctors, risk, mammogram, cancers] | [Artificial Intelligence Is Helping Improve Breast Cancer Detection For Women With Dense Breasts Tech Times Tech Science Business Health Culture Features Buzz Home Health Health Public Living Wellness Artificial Intelligence Is Helping Improve Breast Cancer Detection For Women With Dense Breasts Facebook facebook Twitter twitter Reddit reddit Comment LinkedIn linkedin Email Email WhatsApp What s app RJ Pierce, Tech Times October, am An artificial intelligence algorithm has been helping impro... |
| 10 | 9 | 90 | 9_images_trump_higgins_midjourney | [images, trump, higgins, midjourney, image, putin, fake, context, were, donald] | [Trump arrested Putin jailed Fake AI images spread online Skip to NebraskaContact UsWatch VideoWatch RadarWatch WeatherWeather Shield Request FormWeather CamerasDownload the Weather SchoolSports ResultsNational Politics1011 CaresPure NebraskaPure Nebraska VideoContestsCan Care A VanContact UsMeet the TeamSubmit a News TipCareersAdvertise With Us10 WeatherLocal EverydayHealthy EverydaySeniors EverydayProgramming ScheduleSubmit Photos and VideosCircle Country Music LifestyleGray DC BureauInves... |
| 11 | 10 | 87 | 10_gebru_google_mitchell_timnit | [gebru, google, mitchell, timnit, researcher, company, fired, ethics, her, firing] | [Google fires AI researcher, one of few Black woman in field Stuff.co.nz in TiakiSpotlightStuff NationCartoonsKEA Kids of PlentyTaranakihawke s familyPlay eventsstuff fires AI researcher, one of few Black woman in fieldMatt O Brien14, Dec White Getty ImagesProminent artificial intelligence scholar Timnit Gebru helped improve Google s public image as a company that elevates Black computer scientists and questions harmful uses of AI artificial intelligence scholar Timnit Gebru helped improve G... |
| 12 | 11 | 81 | 11_suicide_safetylit_suicidal_aa | [suicide, safetylit, suicidal, aa, machine, learning, update, doi, bulletin, pdf] | [SafetyLit A comparative study of machine learning techniques for suicide attempts predictive model HOME SEARCH BOOLEAN SEARCH THESAURUS SOURCES AUTHORS WEEKLY UPDATES UPDATE BULLETIN PDF UPDATE BULLETIN WEB ABOUT US MY SAFETYLIT Aa Aa Aa SAFETYLIT WEEKLY UPDATE We compile citations and summaries of about new articles every week. Email Signup RSS Feed HELP Tutorials FAQ CONTACT US Contact info SafetyLit is a service of Search Results Journal Article A comparative study of machine learning te... |
| 13 | 12 | 80 | 12_clearview_facial_recognition_privacy | [clearview, facial, recognition, privacy, enforcement, database, company, law, biometric, shelagh] | [If you re worried about the end of privacy, don t waste your outrage on Clearview AI VentureBeat VentureBeat Homepage Channels GamesBeat AI AR VR Big Data Business Cloud Commerce Dev Enterprise Entrepreneur Esports Marketing Media Mobile PC Gaming Security Social Transportation Got a news tip Press Releases Webinars Advertise VB Lab Guest Posts Events Upcoming Transform GB Summit Sponsor Media Partner Got a news tip Press Releases Webinars Advertise VB Lab Guest Posts Newsletters Special Is... |
| 14 | 13 | 78 | 13_india_flight_air_express | [india, flight, air, express, kozhikode, passengers, plane, airport, delhi, runway] | [Kozhikode plane crash injured passengers discharged from hospitals, says AI Express Select EditionPacificSouth AsiaEast and South East AsiaEurope and Central AsiaCentral AfricaEast AfricaSouthern Africa West AfricaMiddle East and North AfricaNorth America Latin America and Caribbean HOME NEWS RESEARCH LIVE DISCOURSE BLOG OPINION SUBMIT PRESS RELEASE About Career Advertisement Team Partnership Knowledge Partnership Media Partnership Contact Us NEWS RESEARCH LIVE DISCOURSE BLOG OPINION INTERV... |
| 15 | 14 | 78 | 14_number_markets_datanetchng_dataprice | [number, markets, datanetchng, dataprice, canada, market, fed, bank, bloomberg, inflation] | [Nvidia s Surge Sharpens Focus on Hunt for AI Losers BNN Bloomberg Markets Indices Currencies Energy Metals data.symbol group.RICS data.netChng number data.netChng number data data.price number data.price number data.symbol group.RICS data.netChng number data.netChng number data data.price number data.price number Markets As of timeStamp.date timeStamp.time Markets Markets Indices Currencies Energy Metals data.symbol group.RICS data.netChng number data.netChng number data data.price number d... |
| 16 | 15 | 74 | 15_safetylit_aa_update_severity | [safetylit, aa, update, severity, doi, pdf, machine, learning, bulletin, crashes] | [SafetyLit The chance of a traffic collision is predicted using machine learning HOME SEARCH BOOLEAN SEARCH THESAURUS SOURCES AUTHORS WEEKLY UPDATES UPDATE BULLETIN PDF UPDATE BULLETIN WEB ABOUT US MY SAFETYLIT Aa Aa Aa SAFETYLIT WEEKLY UPDATE We compile citations and summaries of about new articles every week. RSS Feed HELP Tutorials FAQ CONTACT US Contact info SafetyLit is a service of Search Results Journal Article The chance of a traffic collision is predicted using machine learning Cita... |
| 17 | 16 | 71 | 16_images_child_abuse_sexual | [images, child, abuse, sexual, porn, deepfake, children, csam, nude, girls] | [Deepfake porn could be a growing problem amid AI race WTTV CBS4Indy Skip to content WTTV CBS4Indy Indianapolis WATCH NOW CBS4 News Sign Up Indianapolis WATCH NOW Sponsored By Toggle Menu Open Navigation Close Navigation Search Please enter a search term. Primary Menu News Indiana News Video Indianapolis Area Crime Consumer News Consumer Alerts Black History Month Your Local Election Headquarters Politics from The Hill NewsNation Now National and World Viral Your Health BestReviews BestRevie... |
| 18 | 17 | 70 | 17_chevron_cnn_oregon_idaho | [chevron, cnn, oregon, idaho, ktvz, bend, local, button, community, weather] | [Chinese police detain man for allegedly using ChatGPT to spread rumors online KTVZ circle arrow Play Button Stop Button chevron right chevron left chevron up search warning chevron left skinny chevron right skinny x clock calendar play button cancel circle user twitter facebook youtube instagram email linkedin Home News Central Oregon Oregon Northwest US World COVID Fire Alert Crime Stoppers KTVZ.COM Polls Special Reports Weather Alerts Interactive Radar Local Forecast Snow Report Road Cond... |
| 19 | 18 | 68 | 18_china_chinese_chips_chip | [china, chinese, chips, chip, beijing, taiwan, us, its, nvidia, alibaba] | [Pull US AI Research Out of China NewsBreakSearch locations, channels, topics, people ... inSign CHANNELSAdd of UsePrivacy PolicyDo Not Sell My InfoHelp CenterAbout Particle Media.RELATED PEOPLEPersonXi JinpingPersonBen SasseIN THIS ARTICLE Google Ai Censorship In China Us Ai Microsoft Exchange Chinese Ccp Georgetown University Facebook Google Ibm Research Asia Lab Party The Joint Chiefs Of Staff Seagate Hikvision American AiYOU MAY ALSO, CNNews of UsePrivacy PolicyDo Not Sell My InfoHelp Ce... |
3. Wordcloud for representation and representation_doc¶
In [77]:
# Flatten the list of words in each representation into a single string and then join all strings
all_representations = ' '.join([' '.join(repr_list) for repr_list in negative_topic_df['Representation']])
# Create a word cloud
wordcloud_rep = WordCloud(background_color='white').generate(all_representations)
# Plotting
plt.figure(figsize=(10, 5))
plt.imshow(wordcloud_rep, interpolation='bilinear')
plt.axis('off')
plt.show()
Representative_Docs (1~11)¶
In [78]:
import matplotlib.pyplot as plt
from wordcloud import WordCloud
# Loop through topics 0 to 9
for topic in range(20):
# Filter the DataFrame for the current topic
topic_data = negative_topic_df[negative_topic_df['Topic'] == topic]
# Extract the first 'Representative_Docs' string for the topic
doc_str = topic_data['Representative_Docs'].iloc[0]
# Explicitly convert to string in case it's not in the correct format
doc_str = str(doc_str)
# Generate word cloud
wordcloud = WordCloud(background_color='white').generate(doc_str)
# Plotting
plt.figure(figsize=(10, 5))
plt.imshow(wordcloud, interpolation='bilinear')
plt.title(f"Word Cloud for Topic {topic}")
plt.axis('off')
plt.show()
3.2. Negative sentiment and topic overtime¶
1. Yearly Analysis¶
1. Aggregate Topic Counts Over Time¶
In [79]:
# Count the frequency of each topic
topic_counts = news_ne['bert_topics'].value_counts()
# Remove topic -1 and get the top 10 topics
top_10_topics = topic_counts.drop(-1).nlargest(10).index
In [80]:
# Filter the dataset
filtered_news_ne = news_ne[news_ne['bert_topics'].isin(top_10_topics)]
In [81]:
# Group by year and topic, and count occurrences
topic_trends = filtered_news_ne.groupby(['year', 'bert_topics']).size().reset_index(name='counts')
2. Pivot the Data for Analysis¶
In [82]:
# Pivot the data
topic_trends_pivot = topic_trends.pivot(index='year', columns='bert_topics', values='counts').fillna(0)
In [83]:
topic_trends_pivot.head()
Out[83]:
| bert_topics | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|---|
| year | ||||||||||
| 2020 | 53 | 4 | 3 | 5 | 2 | 17 | 12 | 1 | 26 | 1 |
| 2021 | 108 | 3 | 3 | 4 | 1 | 10 | 12 | 6 | 14 | 1 |
| 2022 | 5 | 5 | 1 | 7 | 5 | 30 | 5 | 10 | 15 | 2 |
| 2023 | 9 | 130 | 107 | 94 | 92 | 38 | 63 | 73 | 35 | 86 |
3. Plot the Trends¶
In [84]:
# Plot
plt.figure(figsize=(12, 6))
for topic in topic_trends_pivot.columns:
plt.plot(topic_trends_pivot.index, topic_trends_pivot[topic], label=f'Topic {topic}')
plt.xlabel('Year')
plt.ylabel('Topic Counts')
plt.title('Top 10 Topic Trends Over Time')
plt.legend()
plt.show()
4. Detailed Analysis¶
In [85]:
# Example: Print representations of the top N topics
top_topics = topic_trends_pivot.sum().sort_values(ascending=False).head(10).index
for topic in top_topics:
print(f"Topic {topic}: {negative_topic_df.loc[negative_topic_df['Topic'] == topic, 'Representation'].iloc[0]}")
Topic 0: ['jones' 'florida' 'rebekah' 'she' 'warrant' 'state' 'her' 'covid' 'fired' 'scientist'] Topic 1: ['students' 'chatgpt' 'exam' 'cheating' 'university' 'student' 'school' 'schools' 'it' 'education'] Topic 2: ['italy' 'openai' 'data' 'chatgpt' 'italian' 'protection' 'privacy' 'users' 'watchdog' 'ban'] Topic 3: ['humanity' 'ai' 'extinction' 'could' 'risks' 'humans' 'risk' 'nuclear' 'intelligence' 'be'] Topic 4: ['bard' 'google' 'being' 'chatbot' 'or' 'chatgpt' 'may' 'stereotyped' 'people' 'less'] Topic 5: ['fraud' 'financial' 'complyadvantage' 'resistant' 'risk' 'banking' 'transaction' 'transactions' 'banks' 'detection'] Topic 6: ['india' 'delhi' 'opindia' 'modi' 'indian' 'vs' 'february' 'viral' 'in' 'check'] Topic 7: ['fox' 'county' 'ai' 'news' 'public' 'atlanta' 'that' 'of' 'and' 'filefcc'] Topic 8: ['breast' 'cancer' 'radiologists' 'screening' 'mammograms' 'women' 'doctors' 'risk' 'mammogram' 'cancers'] Topic 9: ['images' 'trump' 'higgins' 'midjourney' 'image' 'putin' 'fake' 'context' 'were' 'donald']
In [86]:
import ktrain
from ktrain import text
In [87]:
# Initialize the TransformerSummarizer
ts = text.TransformerSummarizer()
In [88]:
# Step 2: For each topic, retrieve the most representative documents and summarize
for topic_number in range(20): # Replace with your actual number of topics
# Get representative documents for the topic
representative_docs = mod_BERT_neg.get_representative_docs(topic_number)
# Summarize each document
for doc in representative_docs:
summary = ts.summarize(doc)
print(f"Summary of Topic {topic_number+1} Document: {summary}")
Summary of Topic 1 Document: Ex Florida data scientist Rebekah Jones plans to surrenderThe Florida Department of Law Enforcement said Rebekahs Jones has been under investigation since early November. Federal authorities raided her home in December, seizing her computers and other data equipment. Jones was fired from her post in May after she raised questions about Florida s COVID data. Summary of Topic 1 Document: Ex Florida data scientist Rebekah Jones says she plans to surrender to authorities. Federal authorities raided her home in December, seizing her computers and other data equipment. Jones was fired from her post in May after she raised questions about Florida s COVID data. She had been reprimanded several times and was ultimately fired for violating Health Department policy. Summary of Topic 1 Document: Ex Florida data scientist Rebekah Jones says she plans to surrender to authorities. Federal authorities raided her home in December, seizing her computers and other data equipment. Jones was fired from her post in May after she raised questions about Florida s COVID data. She had been reprimanded several times and was ultimately fired for violating Health Department policy. Summary of Topic 2 Document: New York City school officials this week started blocking the impressive but controversial writing tool that can generate paragraphs of human like text. ChatGPT launched on Nov. but is part of a broader set of technologies developed by the San Francisco based startup OpenAI, which has a close relationship with Microsoft. It works like a written dialogue between the AI system and the person asking it questions. Summary of Topic 2 Document: Don't ban ChatGPT in schools. Teach with it Technology News, The Indian Express Sections Search for English. Newsletters Friday, Jan, ePaper Today s Paper Journalism of Courage Subscribe Sign In TrendingDaily CrosswordNew Year OfferHealth SpecialsMovie ReviewsDaily HoroscopeUPSC SpecialRanji Trophy Summary of Topic 2 Document: New York City school officials this week started blocking the impressive but controversial writing tool that can generate paragraphs of human like text. The decision by the largest U.S. school district to restrict the ChatGPT website on school devices and networks could have ripple effects on other schools. Many school districts are still scrambling to figure out how to set policies on if and how it can be used. Summary of Topic 3 Document: Italy temporarily blocks ChatGPT over privacy concerns Today In BC SearchHome Newsletters Subscribe Subscribe Login Logout Support Centre Puzzles Contests News COVID B.C Politics National Politics World News Sports Cannabis Travel Podcasts Video Opinion Classifieds Jobs Business Entertainment Life Weather Obituaries Contact Us Contact Us Black Press FAQ Privacy Policy Terms of use Summary of Topic 3 Document: ChatGPT was launched last November and has gained millions of users worldwide. Italian Data Protection Authority said on Friday March it would block and investigate OpenAI, the US start up that developed ChatGPT, with immediate effect. The announcement has made Italy the first Western country to curb advanced chatbot ChatG PT, which is backed by Microsoft. Summary of Topic 3 Document: Italy temporarily blocks ChatGPT over privacy concerns. U.S. based OpenAI, which developed the chatbot, said late Friday night it has disabled ChatG PT for Italian users at the government s request. The company said it believes its practices comply with European privacy laws. Summary of Topic 4 Document: Artificial Intelligence Raises Risk of Extinction, Experts Say in New Warning. Worries about artificial intelligence systems outsmarting humans and running wild have intensified with the rise of a new generation of highly capable AI chatbots such as ChatGPT. The letter also was signed by experts in nuclear science, pandemics and climate change. Summary of Topic 4 Document: Artificial Intelligence Raises Risk of Extinction, Experts Say in New Warning. Worries about artificial intelligence systems outsmarting humans and running wild have intensified with the rise of a new generation of highly capable AI chatbots such as ChatGPT. The statement doesn t propose specific remedies but some, including Sam Altman, have proposed an international regulator along the lines of the U.N. nuclear agency. Summary of Topic 4 Document: How existential risk became the biggest meme in AI. Ghost stories are contagious. It is also a way to skim over everything that s happening in the present day. It suggests that we haven t seen real or serious harm yet. Concerns about runaway, self improving machines have been around since Alan Turing. Summary of Topic 5 Document: Google has long dominated search, and it s about to massively change it to meet the rising threat of ChatGPT. New AI features could intensify battles with content creators. Publishers could once again complain that their content is being used without fair compensation. Microsoft CEO Satya Nadella said it s a new day for search as he unveiled the revamped Bing with ChatG PT on Tuesday. Summary of Topic 5 Document: Google s AI goof s up BARD AI s mistake during demo, sends Google s stocks tumbling by bn. Google made up a fact about the James Webb Space Telescope, which showcases a vital problem with using AI generators like ChatGPT. The factual error, plus ChatG PT Bing s launch, sent Google s stock tumbling. Summary of Topic 5 Document: MailOnline asks ChatGPT to come up with a stereotype for residents in all UK counties prepare to be offended. It called people from Essex shallow, while calling Cornish people lazy and Durham residents were labeled uncouth. The cutting edge bot labeled Yorkshiremen as rude while Londoners were slammed for their arrogance in the nationwide analysis. Summary of Topic 6 Document: Resistant AI and ComplyAdvantage Launch AI Transaction Monitoring Solution To Combat Fraud and Money Laundering. Holvi, the digital banking service for small businesses, is among the initial group of customers to implement the AI driven solution to manage their financial crime risk. According to the United Nations, the estimated amount of money laundered globally in one year is of global GDP, or billion trillion US dollars. Summary of Topic 6 Document: Resistant AI and ComplyAdvantage Launch AI Transaction Monitoring Solution To Combat Fraud and Money Laundering. Holvi, the digital banking service for small businesses, is among the initial group of customers to implement the AI driven solution to manage their financial crime risk. According to the United Nations, the estimated amount of money laundered globally in one year is of global GDP, or billion trillion US dollars. Summary of Topic 6 Document: Resistant AI and ComplyAdvantage Launch AI Transaction Monitoring Solution To Combat Fraud and Money Laundering. Holvi, the digital banking service for small businesses, is among the initial group of customers to implement the AI driven solution to manage their financial crime risk. Financial crime is a multi trillion dollar problem. Summary of Topic 7 Document: N Nvidia CEO Jensen Huang meets PM Modi, highlights growing partnership in AI and India s young talent pool. Udhayanidhi Stalin should be declared the leadership face of the I.N.D.A. alliance, at least he is clear and honest about his priorities. How a seemingly small Netherlands website spreads anti Hindu propaganda and has links with ISIS quoted journalist, LeT linked org, leftists. Summary of Topic 7 Document: Elon Musk Tried To Take Over OpenAI in, but Failed After Being Rejected by Sam Altman and Other Founders LatestLY Advertisement Live Breaking News 7th Pay Commission Good News Today Centre Approves Release of Additional Installment of DA to Government Employees and Dearness Relief to Pensioners Due From January Jammu and Kashmir One Dead, Three Others Injured After Explosion At Scrap Factory in Bari Brahmana Rahul Gandhi Disqualification BJP, Centre Have No Role in Court s Decision on Congress Leader, Says Parliamentary Board Member Sudha Yadav Ramadan Time Table Sehri and Iftar Timings for 3rd Roza of Ramzan on March in Mumbai, Lucknow, Delhi, Kolkata and Other Cities Summary of Topic 7 Document: World News First ChatGPT Arrest in China over Fake Train Crash News LatestLY Advertisement Live Breaking News The Kerala Story Banned in West Bengal TMC Government Bans Adah Sharma s Film To Avoid Incident of Hatred and Violence, Says CM Mamata Banerjee Balcony Sex in Public Goes Terribly Wrong for Randy Couple As Half Naked Woman Falls on Car Roof, Old Video Goes Viral KL Rahul Ruled Out of WTC Final Against Australia Ishan Kishan Named As His Replacement in ICC World Test Championship Final Squad. Summary of Topic 8 Document: Misinformation machines AI chatbots can spew falsehoods, even accuse people of crimes they never committed. Severe Thunderstorm Warning until SAT PM EDT, Frederick County, Montgomery County, Carroll County. Google CEO Sundar Pichai told Minutes that he and other tech titans don t fully understand how AI works. Summary of Topic 8 Document: Misinformation machines AI chatbots can spew falsehoods, even accuse people of crimes they never committed To DoContests More Watch Live Expand Collapse search site News Local NewsNational NewsWorld For ChangeFOX News SundayWeather FOX Weather AppForecastSchool ClosingsLive Weather CamerasTrafficFOX WeatherSports Shayne WellsGarden GuyRecipesMoney Personal MarketSmall FOX Shows the Jason ShowFOX Good DayEnough SaidVikings Gameday LiveThe PJ Fleck ShowFOX Sports NowThe Jason Show Swag ShopThe FOX StoreRegional News Milwaukee News FOX NewsChicago News FOX ChicagoDetroit News FOX DetroitAbout Us Contact at FOX 9What s On FOXAdvertiseFCC Public FileFCC ApplicationsStay Connected River Flood Summary of Topic 8 Document: Misinformation machines AI chatbots can spew falsehoods, even accuse people of crimes they never committed LiveNewsGood Day LAContestsTV LinksArmenian Heritage More Watch Live Expand Collapse search site News Local NewsCalifornia NewsNational News AppFOX News SundayWeather ForecastRadarAir Quality PreparednessSevere WeatherGood Day LA GuestsMeet the NewsSeen on TV LinksPolitics California PoliticsThe Issue Is Joe BidenKamala HarrisGavin NewsomGeorge GascónDonald TrumpSports Football ClubAngel City Football FOX OriginalsLost Angeles City of HomelessCommunity ChampionsSave Our StreetsWhen Magic Shocked the WorldDestination EducationGet InspiredRising UpThe Issue is InvestigationsTrue Crime FilesIn DepthWednesday s ChildCulture Kobe Summary of Topic 9 Document: Artificial Intelligence Is Helping Improve Breast Cancer Detection For Women With Dense Breasts. Using artificial intelligence, the researchers were able to look at a total of 9,200 MRI scans of dense breasts. The artificial intelligence flagged of all the scans with lesions, and dismissing of those without lesions, all without missing any cancers. Summary of Topic 9 Document: Artificial intelligence can be trained to outperform humans when it comes to catching breast tumours on mammograms. New study at Google and several universities are working on an artificial intelligence AI model aimed at improving the accuracy of mammography screening. Overall, the model reduced false positive and false negative results. The improvement was greater in the United States. Summary of Topic 9 Document: Google s AI system can beat doctors at detecting breast cancer WQAD.com GoSearch Watch Now News at News TV Schedule Autos Search Contact Us WQad.com Menu News GMQC Bridges Traffic Life in the QC On Air Contests Deals Brewed Sports Podcasts Weather Low High Tue Wed Thu See complete forecast Google s AI System can beat Doctors at detecting Breast Cancer. Summary of Topic 10 Document: Trump arrested Putin jailed Fake AI images spread online. Experts warn the images are harbingers of a new reality waves of fake photos and videos flooding social media after major news events. The highly detailed, sensational images have inundated Twitter and other platforms, amid news that Trump faces possible criminal charges and the International Criminal Court has issued an arrest warrant for Putin. Summary of Topic 10 Document: Trump arrested Putin jailed Fake AI images spread online. Misinformation experts warn the images are harbingers of a new reality waves of fake photos and videos flooding social media after major news events. It does add noise during crisis events. You start to lose trust in the system and the information that you are getting. Summary of Topic 10 Document: Trump arrested Putin jailed Fake AI images spread online. Experts warn the images are harbingers of a new reality waves of fake photos and videos flooding social media after major news events. The highly detailed, sensational images have inundated Twitter and other platforms in recent days, amid news that Trump faces possible criminal charges and the International Criminal Court has issued an arrest warrant for Putin. Summary of Topic 11 Document: Google fires AI researcher, one of few Black woman in field Stuff.co.nz in TiakiSpotlightStuff NationCartoonsKEA Kids of PlentyTaranakihawke s familyPlay eventsstuff fires AI scholar Timnit Gebru. She was pushed out of the company this week in a dispute over a research paper examining the societal dangers of AI. Summary of Topic 11 Document: Hundreds of Google workers condemn firing of AI scientist Timnit Gebru. More than 1,000 researchers also sign letter after Black expert on ethics says Google tried to suppress her research on bias. Google has maintained that Gebu resigned. Nearly Google employees and over 1,100 supporters from academia and civil society signed a letter of protest. Summary of Topic 11 Document: Google AI Team Demands Ousted Black Researcher Be Rehired And Promoted. Timnit Gebru was one of the only Black research scientists at Google. She was fired after a dispute over an academic paper and months of speaking out about the need for more women and people of color at the tech giant. Summary of Topic 12 Document: SafetyLit A comparative study of machine learning techniques for suicide attempts predictive model. Current suicide risk assessments for predicting suicide attempts are time consuming, of low predictive value and have inadequate reliability. An ensemble predictive models outperformed the single predictive models. History of suicide attempt, religion, race, suicide ideation and severity of clinical depression are useful factors for prediction of suicide attempts. Summary of Topic 12 Document: SafetyLit Artificial intelligence and suicide prevention a systematic review of machine learning investigations. Suicide is a leading cause of death that defies prediction and challenges prevention efforts worldwide. Artificial intelligence AI and machine learning ML have emerged as a means of investigating large datasets to enhance risk detection. We report key findings and central limitations in the use of AI ML frameworks to guide additional research. Summary of Topic 12 Document: SafetyLit A machine learning approach for predicting wage workers suicidal ideation HOME SEARCH BOOLEAN SEARCH THESAURUS SOURCES AUTHORS WEEKLY UPDATES UPDATE BULLETIN PDF UPDATE Bulletin WEB ABOUT US MY SAFETYLIT Aa Aa aa A a SAFETYIT WEEKLY UPDATE We compile citations and summaries of about new articles every week. Summary of Topic 13 Document: If you re worried about the end of privacy, don t waste your outrage on Clearview AI. The way it operates may be insensitive or even horrifying, but save your questions for the businesses and governments working with Clearview. People deserve answers to the questions Senator Markey asks about the extent of the data breach. Summary of Topic 13 Document: Clearview AI Reports Entire Client List Was Stolen SoylentNews is people Navigation Privacy Policy About FAQ Journals Topics Authors Search Polls Hall of Fame Submit Story Subs Queue Buy Gift Sub Create Account Log In Sections Privacy Policy Twitter IRC Wiki Who s Who Bug List Dev Server Log In Privacy Policy Log In Nickname Password Public Terminal Create Account Retrieve Password Gift a Subscription Why Gift Related Links Privacy Policy SoylENTNews fliptop got unauthorized access to its list of customers a front page story Canadian Privacy Commissioners to Investigate Creepy Facial Recognition Firm Clearview AI. Clearview App Lets Strangers Find Your Name, Info with Snap of a Photo, Report Says. Summary of Topic 13 Document: Clearview AI to Stop Selling Controversial Facial Recognition App to Private Companies. Vermont Sues Clearview, Alleging Oppressive, Unscrupulous Practices. Clearview AI Reports Entire Client List Was Stolen Canadian Privacy Commissioners to Investigate Creepy Facial recognition firm. Summary of Topic 14 Document: Kozhikode plane crash injured passengers discharged from hospitals, says AI Express. Air India Express has only B737 aircraft in its fleet. The airline said its contract service provider Kenyon International has started the process of search and retrieval of the baggage belonging to the crew and passengers who were in the aircraft. Summary of Topic 14 Document: India News AI Express Flight Overshot Runway, No Fire During Landing Aviation Ministry LatestLY Live Breaking News Deeply Saddened to Hear About Accident of Air India Express Flight IX1344 Upon Landing at Kozhikode Airport, Says International Air Transport Association Live News Breaking And Coronavirus Updates on August. How to Watch Juventus vs Lyon, UEFA Champions League Live Streaming Online in India Get Free Live Telecast of JUV vs LYN Football Score Updates on TV How to watch Manchester City vs Real Madrid, UEFA Football Score updates on TV DC FanDome Trailer Dwayne Johnson Teases With Black Adam Look, Fans Are Beyond Thrilled. Summary of Topic 14 Document: Kerala Plane Crash AI Express Says Relief Flights Arranged To Assist Passengers. Air India Express flight from Dubai with on board overshot the tabletop runway at Kozhikode airport on Friday night while landing in heavy rains. The aircraft fell into a valley feet below and broke into two, killing at least people. Summary of Topic 15 Document: N Nvidia s Surge Sharpens Focus on Hunt for AI Losers BNN Bloomberg Markets Indices Currencies Energy Metals data. Try one of these result.description result.ric More Results BNN Live Video Shows Market Call Markets AI Investing Personal Finance Real Estate Company News Commodities Economics Politics Technology Opinion ETFs International Market Call Schedule. Summary of Topic 15 Document: Legal Data Firm Makes Push to Introduce AI to Lawyers Work. Canadian real estate prices bottomed out Job Gains Top Forecasts in Canada Unemployment Holds at Ireland Weighs Mortgage Relief to Help Ease Pain from Rate Hikes. More Chinese Banks Cut Deposit Rates as Margins Shrink. Summary of Topic 15 Document: Microsoft Bets on AI as the Next Wave Amid Slowing Growth BNN Bloomberg Markets Indices Currencies Energy Metals data. Try one of these result.description result.ric More Results BNN Live Video Shows Market Call Markets Investing Personal Finance Real Estate Company News Commodities Economics Politics Technology Bloomberg News Wire Opinion ETFs Executive Pursuits International Market Call Schedule. Summary of Topic 16 Document: SafetyLit The chance of a traffic collision is predicted using machine learning. One of the main causes of fatalities, disabling injuries, and hospitalisation in the nation continues to be traffic accidents. The study s goal is to estimate the likelihood of a collision by looking at data that drivers would already be aware of, such as vehicle type, age, gender. Summary of Topic 16 Document: SafetyLit Machine learning approaches to traffic accident analysis and hotspot prediction HOME SEARCH BOOLEAN SEARCH THESAURUS SOURCES AUTHORS WEEKLY UPDATES UPDATE BULLETIN PDF UPDATE Bulletin WEB ABOUT US MY SAFETYLIT Aa Aa aa A a SAFETYLit WEEKLY UPDATE We compile citations and summaries of about new articles every week. Email Signup RSS Feed HELP Tutorials FAQ CONTACT US Contact info SafetyLit is a service of Search Results Journal Article Machine learning approach to traffic accidents. Summary of Topic 16 Document: SafetyLit Using machine learning models to forecast severity level of traffic crashes by R Studio and ArcGIS Home SEARCH BOOLEAN SEARCH THESAURUS SOURCES AUTHORS WEEKLY UPDATES UPDATE BULLETIN PDF UPDATE Bulletin WEB ABOUT US MY SAFETYLIT Aa Aa a Aa SAFETYIT WEEKLY UPDATE We compile citations and summaries of about new articles every week. SafetyLit is a service of Search Results Journal Article using machine learning ML techniques. Summary of Topic 17 Document: Deepfake porn could be a growing problem amid AI race. Australian Noelle Martin found deepfake porn of herself years ago when out of curiosity one day she used Google to search an image of herself. Martin says she doesn't know who created the fake images, or videos of her engaging in sexual intercourse that she would later find. Summary of Topic 17 Document: Deepfake porn could be a growing problem amid AI race. Porn created using the technology first began spreading across the internet several years ago when a Reddit user shared clips that placed the faces of female celebrities on the shoulders of porn actors. The problem, experts say, grew as it became easier to make sophisticated and visually compelling deepfakes. Noelle Martin, of Perth, Australia, has experienced that reality. Summary of Topic 17 Document: Deepfake porn could be a growing problem amid AI race. Deepfakes are videos and images that have been digitally created or altered with artificial intelligence or machine learning. Porn created using the technology first began spreading across the internet several years ago. Some AI models say they re already curbing access to explicit images. Summary of Topic 18 Document: Chinese police detain man for allegedly using ChatGPT to spread rumors online KTVZ.COM Polls Special Reports Weather Alerts Interactive Radar Local Forecast Snow Report Road Conditions Weather Webcams Sports Prep Scoreboard Basketball Videos Galleries Livestream Newscasts LivestREAM Special Coverage Local Videos Photo Galleries Community Cares For Kids Community Billboard Community Links Let s Talk One Class At a Time Pay it Forward Lifestyle Now Hiring House Home Entertainment Events Calendar Health Money Pets Pump Patrol Technology Contests Contests Pet Pics Sweepstakes Summary of Topic 18 Document: things to know for May FBI, New Mexico shooting, Gas prices, Debt limit, AI KTVZ. Extended boycott disqualifies GOP senators, independent in Oregon Senate from being reelected Bend s downtown parklet program stays in place past pandemic, gets mixed reviews from businesses. Springtime pollen especially bad in Central Oregon this year. Summary of Topic 18 Document: things to know for May FBI, New Mexico shooting, Gas prices, Debt limit, AI. Face masks have largely disappeared from places like grocery stores and schools in the third year of the Covid pandemic. FBI Special counsel John Durham released his final report on Monday in which he casts doubt about the FBI s decision to launch a full investigation into connections between Donald Trump s campaign and Russia. Summary of Topic 19 Document: Pull US AI Research Out of China NewsBreakSearch locations, channels, topics, people... inSign CHANNELSAdd of UsePrivacy PolicyDo Not Sell My InfoHelp CenterAbout Particle Media. 0See allWhat are your thoughts PostCommunity Policy14. China is on the march.... 763725SharePOTUS. Summary of Topic 19 Document: Former Pentagon Official Says China Has Overtaken US In AI Race NewsBreakSign ArtTV SeriesBooks DanceBehind Viral VideosPerforming ArtsTV MusicHip. HealthHealth ServicesMental HealthDiseases s HealthCancerFood SportsPremier DrinksPetsBeauty SafetyPublic SafetyAccidentsLaw EnforcementTraffic AdviceFamily RentLabor IssuesTrouble ScienceEarth NationsMiddle locations, channels, topics, people... inGAMINGbible Follow1K Followers3K Post58K brings you the latest video game news, reviews of the most exciting releases, and interviews with the industry s biggest names. Summary of Topic 19 Document: World Artificial Intelligence Conference kicked off in China s Shanghai News BreakSearch locations, channels, topics, people... inSign CHANNELSAdd of UsePrivacy PolicyDo Not Sell My InfoHelp CenterAbout Particle Media. 0See allWhat are your thoughts PostCommunity Policy Summary of Topic 20 Document: CryptoRom Scammers Add AI Chat Tool, Like ChatGPT, and Fake Hacks on Crypto Accounts to Their Toolset, Sophos Finds. In, investment fraud caused the highest losses of any scam reported by the public to the US FBI s Internet Crimes Complaint Center IC3. Summary of Topic 20 Document: Pig butchering scams are using AI tools, says Sophos. Sha Zhu Pan Scam Uses AI Chat Tool to Target iPhone and Android Users. Scammers also expanded their coercion tactics by telling victims their crypto accounts were hacked and more upfront money is needed. Sophos X Ops found seven fake cryptocurrency investment apps in the official Google Play and Apple App stores. Summary of Topic 20 Document: CryptoRom Scammers Add AI Chat Tool, Like ChatGPT, and Fake Accessibility Skip TopNav. Scammers Also Snuck New Fake Apps Into the Apple and Google Play Stores. Investment fraud caused the highest losses of any scam reported by the public to the FBI s Internet Crimes Complaint Center IC3.